Bayesian learning for neural networks: an algorithmic survey

Abstract The last decade witnessed a growing interest in Bayesian learning. Yet, the technicality of topic and multitude ingredients involved therein, besides complexity turning theory into practical implementations, limit use learning paradigm, preventing its widespread adoption across different fields applications. This self-contained survey engages introduces readers to principles algorithms Learning for Neural Networks. It provides an introduction from accessible, practical-algorithmic perspective. Upon providing general Networks, we discuss present both standard recent approaches inference, with emphasis on solutions relying Variational Inference Natural gradients. We also manifold optimization as state-of-the-art approach examine characteristic properties all discussed methods, provide pseudo-codes their implementation, paying attention aspects, such computation